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DETAILED ACTION 

Response to Arguments 

1 . Regarding claims 1 , the examiner respectfully disagrees. The prior art of record 
teaches in part "replaying said operations and said accompanying audio in said 
computer environment, said operations resulting from processing of recorded user 
inputs". The "operations resulting from processing of recorded user inputs" are taught 
by Drews, wherein Drews teaches recording user input, such as audio and a cursor 
control device (see column 5, lines 37-57, and column 7, line 12 - column 8, line 52). 

Although the claims are interpreted in light of the specification, limitations from 
the specification are not read into the claims. The claim limitations are afforded the 
broadest reasonable interpretation, and the prior art as interpreted reads on these 
features. It is noted that the specification teaches "Event Recording" on page 7, in 
paragraphs 0050 and 0051, wherein event recording "includes all user inputs" during 
recording and event recording includes "recorded initial conditions". However, this 
disclosure is not in the claim limitations and the "exemplary embodiment of the 
invention" is not read into the current claim language. 

2. Regarding claims 2-6, 9, 10, 12-14, 27, 28, and 30-40, see the preceding 
argument with respect to claim 1 and the following rejections under 35 USC 103. 

Claim Rejections - 35 USC § 103 

3. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 



Application/Control Number: 10/770,935 
Art Unit: 2614 



Page 3 



4. Claims 1-6, 9, 10, 12-24, 27, 28, and 30-40 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Drews et al. (previously cited and hereinafter Drews), US 
5,768,607 A, in view of Baumgartner et al. (previously cited and hereinafter 
Baumgartner), US 5,642,171 A. 

5. Regarding claim 1 , Drews teaches a method for synchronizing operations in a 
computer environment with accompanying audio (see column 3, line 15 - column 4, line 
2), said method comprising: 

replaying said operations and said accompanying audio in said computer environment, said 
operations resulting from processing of recorded user inputs (see column 5, lines 37-57, and 
column 7, line 12 - column 8, line 52); 

creating a synchronization point at a common point in said replaying of said operations and said 
accompanying audio (see column 13, lines 30-36); 

associating said synchronization point with said accompanying audio, said synchronization point 
providing a reference point to substantially synchronize said accompanying audio when said operations 
are replayed in a replay computer environment using said recorded user inputs (see column 1 3, lines 
30-36); 

detecting said synchronization point during a subsequent replay of said operations and said 
accompanying audio in said replay computer environment, said subsequent replay involving another 
processing of said recorded user inputs (see column 14, lines 4-26); 

comparing said synchronization point with a time value associated with said another processing 
of said recorded user inputs (see column 14, lines 4-48) ; 

selectively pausing said subsequent replay of said accompanying audio if a difference between 
said synchronization point and said time value exceeds a predefined amount so that said subsequent 
replay of said operations can catch up to said accompanying audio; and 

resuming said subsequent replay of said accompanying audio if a difference between said 
synchronization point and a current time value does not exceed a second predefined amount, said current 
time value being associated with said another processing of said recorded user inputs. 

Drews teaches a method, wherein operations with accompanying audio is synchronized 

in a computer environment. However, Drews does not teach pausing accompanying 

audio if the operations (i.e. recorded user inputs) fall behind the synchronization, so that 

they video can catch up to the audio portion of the recording. 

Baumgartner teaches a method of synchronizing audio and video streams in a 



computer environment (see abstract). Specifically, Baumgartner teaches an 
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environment, wherein the processing capabilities of the system are too slow (see 
column 5, lines 1 8-27). It is not hard for one of ordinary skill in the art at the time of the 
invention to appreciate that a video processor may have the same deficiencies. 
Baumgartner teaches pausing the audio if the audio is too far ahead of the video 
playback (see column 7, lines 16-23). Specifically, Baumgartner teaches the selectively 
pausing and resuming steps (see figure 5A, steps 518, 520, 522, and 524 and column 
14, lines 1-26). It would have been obvious for one of ordinary skill in the art at the time 
of the invention to combine the teachings of Drews and Baumgartner for the purpose of 
synchronizing audio and video (i.e. recorded user input) on slower computer hardware. 

6. Regarding claim 2, the further limitation of claim 1 , see the preceding argument 

with respect to claim 1 . The combination teaches the method of claim 1 , wherein 

creating of said synchronization point includes creating said synchronization point in response to 
a user command (see Drews, column 13, lines 30-36, wherein a user recording sound and 
cursor movement causes the creation of synchronization points). 

7. Regarding claim 3, the further limitation of claim 1 , see the preceding argument 

with respect to claim 1 . The combination teaches the method of claim 1 , wherein 

said common point is at a point in time where there is no audio output during said replaying of 
said accompanying audio (see Drews, column 13, lines 37-45 and figure 11a, wherein 
common points are created at the beginning of recording, when there is no sound). 

8. Regarding claim 4, the further limitation of claim 1 , see the preceding argument 
with respect to claim 1 . The combination teaches the method of claim 1 , further 
comprising 

obtaining a current time value associated with said processing of said recorded user inputs, said 
current time value corresponding to said synchronization point (see Baumgartner, figures 5a and 
5b and column 13, lines 23-67, wherein the current time value is implied to be a current 
audio or video time index). 
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9. Regarding claim 5, the further limitation of claim 1 , see the preceding argument 
with respect to claim 1 . In the combination, Baumgartner teaches different ways of 
storing the video and the audio (see column 10, lines 30-39). Baumgartner does not 
appear to teach a method of claim 1, wherein the synchronization point is saved in the 
first file with the recorded audio, but it would have been obvious for one of ordinary skill 
in the art at the time of the invention to place the synchronization points in either file, or 
a single file with interleaved data. It would have been an obvious design choice. 

1 0. Regarding claim 6, the further limitation of claim 1 , see the preceding argument 
with respect to claim 1 . The combination teaches the features of claim 1 , and makes 
these features obvious. In the combination, Drews teaches the editing of the user 
commands, or drawings, and editing of the recorded sound (see column 10, lines 30- 
40). It is inferred by the editing, or appending, of the recording will change a time value 
of the synchronization points. Drews teaches a timeline (see figure 11a, 11b, and 1 2). 
It would have been obvious for one of ordinary skill in the art at the time of the invention 
to allow direct editing of the timeline by moving a positional marker on the timeline as is 
known in video editing. 

1 1 . Regarding claim 9, the further limitation of claim 1 , see the preceding argument 

with respect to claim 1 . The combination teaches the method of claim 1 , wherein 

said second predefined amount equals said predefined amount (see Baumgartner, column 
14, lines 1-26 teaches one embodiment, wherein the predefined amounts are different). 

In the combination, Baumgartner teaches a preferred embodiment, wherein it is obvious 

to change the predetermined amounts according to different applications. It would have 

been obvious for one of ordinary skill in the art at the time of the invention to use the 
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same predefined amounts or thresholds to pause or resume the audio in order to make 
the comparison of the current time value with the synchronized playback value easier. 

12. Regarding claim 10, see the preceding argument with respect to claim 1 . The 
combination teaches the method of claim 1, which reads on the method of claim 10. 

13. Regarding claim 12, the further limitation of claim 10, seethe preceding 
argument with respect to claim 9. The combination makes obvious the method of claim 
10 with these additional features. 

14. Regarding claim 13, the further limitation of claim 10, see the preceding 
argument with respect to claim 6. The combination makes obvious the method of claim 
10 with these additional features. 

15. Regarding claim 14, the further limitation of claim 10, seethe preceding 
argument with respect to claim 1 . The combination teaches the features of claim 10 
and these additional features. 

16. Regarding claim 15, the further limitation of claim 14, seethe preceding 
argument with respect to claim 2. The combination teaches the features of claim 14 
and these additional features. 

17. Regarding claim 16, the further limitation of claim 14, see the preceding 
argument with respect to claim 3. The combination teaches the features of claim 14 
and these additional features. 

18. Regarding claim 17, the further limitation of claim 14, seethe preceding 
argument with respect to claim 5. The combination teaches the features of claim 14 
and these additional features. 
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19. Regarding claim 18, the further limitation of claim 14, seethe preceding 
argument with respect to claim 6. The combination makes obvious the method of claim 
14 with these additional features. 

20. Regarding claim 19, see the preceding argument with respect to claim 1 . The 
combination teaches the method of claim 1 , which reads on the storage medium 
readable by a computer, tangibly embodying a program of instructions executable by 
said computer to perform method steps for synchronizing operations in a computer 
environment with accompanying audio, said method comprising the features of claim 1 . 

21 . Regarding claim 20, the further limitation of claim 19, see the preceding 
argument with respect to claim 2. The combination teaches the features of claim 2, 
which reads on the features of claim 20. 

22. Regarding claim 21 , the further limitation of claim 1 9, see the preceding 
argument with respect to claim 3. The combination teaches the features of claim 3, 
which reads on the features of claim 21 . 

23. Regarding claim 22, the further limitation of claim 19, see the preceding 
argument with respect to claim 4. The combination teaches the features of claim 4, 
which reads on the features of claim 22. 

24. Regarding claim 23, the further limitation of claim 1 9, see the preceding 
argument with respect to claim 5. The combination teaches the features of claim 4, 
which reads on the features of claim 23. 
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25. Regarding claim 24, the further limitation of claim 1 9, see the preceding 
argument with respect to claim 6. The combination makes obvious the method of claim 
19 with these additional features. 

26. Regarding claim 27, the further limitation of claim 19, see the preceding 
argument with respect to claim 9. The combination teaches the features of claim 4, 
which reads on the features of claim 27. 

27. Regarding claim 28, see the preceding argument with respect to claim 1 . The 
combination teaches the method of claim 1 , which reads on the storage medium 
readable by a computer, tangibly embodying a program of instructions executable by 
said computer to perform method steps for synchronizing operations in a computer 
environment with accompanying audio, said method comprising the features of claim 
10. 

28. Regarding claim 30, the further limitation of claim 28, see the preceding 
argument with respect to claim 9. The combination makes obvious the storage medium 
of claim 28 with these additional features. 

29. Regarding claim 31 , the further limitation of claim 28, see the preceding 
argument with respect to claim 6. The combination makes obvious the method of claim 
28 with these additional features. 

30. Regarding claim 32, the further limitation of claim 28, see the preceding 
argument with respect to claim 1 . The combination teaches the features of claim 28 
and these additional features. 
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31 . Regarding claim 33, the further limitation of claim 32, see the preceding 
argument with respect to claim 2. The combination teaches the features of claim 32 
and these additional features. 

32. Regarding claim 34, the further limitation of claim 32, see the preceding 
argument with respect to claim 3. The combination teaches the features of claim 32 
and these additional features. 

33. Regarding claim 35, the further limitation of claim 32, see the preceding 
argument with respect to claim 5. The combination teaches the features of claim 32 
and these additional features. 

34. Regarding claim 36, the further limitation of claim 32, see the preceding 
argument with respect to claim 6. The combination makes obvious the method of claim 
32 with these additional features. 

35. Regarding claim 37, see the preceding argument with respect to claim 16. The 
combination teaches a method with the combined features of claims 10, 14, and 16. 

36. Regarding claim 38, see the preceding argument with respect to claim 37. The 
combination teaches the method of claim 16, which reads on the storage medium 
readable by a computer, tangibly embodying a program of instructions executable by 
said computer to perform method steps for synchronizing operations in a computer 
environment with accompanying audio, said method comprising the features of claim 
37. 

37. Regarding claim 39, see the preceding argument with respect to claim 17. The 
combination teaches a method with the combined features of claims 10, 14, and 17. 
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38. Regarding claim 40, see the preceding argument with respect to claim 39. The 
combination teaches the method of claim 17, which reads on the storage medium 
readable by a computer, tangibly embodying a program of instructions executable by 
said computer to perform method steps for synchronizing operations in a computer 
environment with accompanying audio, said method comprising the features of claim 
39. 

Conclusion 

39. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Owen, US 5,585,583 A, (previously cited) teaches mixing a student's musical 
instrument with a synchronized audio/video playback of an instructional musical piece 
(abstract, column 3, line 57 - column 4, line 9, column 6, lines 43-59, column 7, lines 15- 
26, column 7, line 56 - column 8, line 16 and column 13, lines 48); 

Kuribayashi, US 5,615,189 A, (previously cited) teaches synchronization of a 
word clock in MIDI circuits (abstract and column 1, line 50 - column 2, line 20); 

Berstis et al., US 5,903,266 A, (previously cited) teaches audio instructions 
detailing various steps of connecting equipment (column 5, line 48 - column 8, line 22 
and figures 3 and 4); 

Falcon et al., US 6,632,094 B1 , (previously cited) teaches synchronized text 
highlighting with audio narration for improving reading skills (abstract and column 8, 
lines 23-26); 
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Sastry et al., US 6,687,877 B1 , (previously cited) teaches content annotation 
synchronized with user actions and recorded audio (column 5, lines 39-61); 

Blume, US 6,915,103 B2, (previously cited) teaches an audio broadcast system 
to read an audio track corresponding to the position and speed of a handheld stylus 
(column 3, line 32-56 and figures 1-5); 

Lilienthal, US 6,933,928 B1 , (previously cited) teaches a paperless book with 
visual text that advances as the audio text is finished (column 3, lines 43-50); and 

Gaddy et al., US 7,228,189 B1, (previously cited) teaches a method for 
synchronizing a streaming music file with a locally captured audio input (abstract, 
column 7, line 39 - column 8, line 36, and figures 3a and 3b). 

40. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DANIEL R. SELLERS whose telephone number is 
(571)272-7528. The examiner can normally be reached on Monday to Friday, 9am to 
5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Curtis Kuntz can be reached on (571)272-7499. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Daniel R. Sellers/ 
Examiner, Art Unit 2614 
/CURTIS KUNTZ/ 

Supervisory Patent Examiner, Art Unit 2614 



